当研究室では、マーケティングにおいて顧客の商品への興味を推定するための顧客の姿勢推定を行っています。 姿勢推定の場合、多くの深層学習アプローチは主に関節の検出に焦点を当てています。 しかし、推定したい人物の一部が他者や物によって隠れているオクルージョン状態のとき、関節の検出がうまくいかない場合があります。 また、関節情報だけでなく体の境界や体の向きなどの情報を考慮することで、姿勢推定が向上することが考えられます。 そこで、マルチタスク戦略を使い、複数の情報を考慮した深層学習モデルを提案します。 具体的には、Mask-RCNNに基づき、関節情報、体の境界、体の向き、オクルージョン状態の４つを統合的に推定します。

For years, behavior understanding has been a hot topic in the field of computer vision. As an important part of human behavior understanding, pose estimation has attracted lots of interests. Recently, deep learning methods, such as Mask R-CNN, have achieved much better performance for computer vision tasks than that of traditional approaches, as deep neural network can find representative features efficiently. For pose estimation, most of the deep learning approaches mainly focus on the joint feature. However, this feature is not sufficient, especially when the pose is occluded or not intact. In fact, many features other than joint can also contribute to pose estimation, such as body boundary, body orientation and visibility condition. By adopting multi-task strategy, these features can be efficiently combined inside deep learning model for pose estimation. In this paper, we present a multi-task pose estimation approach to deal with human behavior understanding. Our deep learning model is based on Mask-RCNN, of which the output contains 4 tasks: human keypoint prediction, body segmentation, orientation prediction and mutual occlusion detection.